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The ordering of the classical Heisenberg antiferromagnet on the triangular lattice with the the 
bilinear-biquadratic interaction is studied by Monte Carlo simulations. It is shown that the model 
exhibits a topological phase transition at a finite-temperature driven by topologically stable vortices, 
while the spin correlation length remains finite even at and below the transition point. The relevant 
vortices could be of three different types, depending on the value of the biquadratic coupling. 
Implications to recent experiments on the triangular antiferromagnet NiGa2S4 is discussed. 



Antiferromagnetic (AF) Heisenberg model on the two- 
dimensional (2D) triangular lattice has been studied 
extensively as a typical example of geometrically frus- 
trated magnets. Inspired by recent experiments on a 
variety of triangular magnets, including NiGa2S4[l], 0] 
and NaCrC^jll], renewed interest has now arisen in this 
model. The triangular-lattice Heisenberg antiferromag- 
net (AF) with the nearest-neighbor bilinear exchange is 
known to exhibit a magnetic long-range order (LRO) at 
T = 0, the so-called 120° structure, in either case of quan- 
tum 5 = 1/2 or classical 5 — oo spin. Because of the 
two-dimensionality of the lattice, the AF LRO is estab- 
lished only at T — 0, while the associated spin correlation 
length diverges exponentially toward T = 0. 

Some time ago, it was demonstrated by Kawamura 
and Miyashita (KM) that the triangular Heisenberg AF 
bears a topologically stable point defect characterized by 
a two-valued topological quantum number, Z2 vortex, in 
contrast to its unfrustrated counterpart j4|. Existence 
of such a vortex has become possible owing to the non- 
collinear nature of the spin order induced by frustration. 
KM suggested that the triangular Heisenberg AF might 
exhibit a genuine thermodynamic transition at a finite 
temperature associated with the condensation (binding- 
unbinding) of Z2 vortices. This topological transition 
is of different character from the standard Kosterlitz- 
Thouless (KT) transition in that the spin correlation 
length does not diverge even at and below the transition 
point and the spin correlation in the low-temperature 
phase decays exponentially. The topological transition 
occurs between the two spin paramagnetic states. 

On experimental side, recent data by Nakatsuji et al on 
the S=l triangular Heisenberg AF NiGa2S4 are of partic- 
ular interest: While no magnetic LRO is observed to low 
temperature, the low-temperature specific heat exhibits a 
T 2 behavior, suggesting the existence of Goldstone modes 
associated with a broken continuous symmetry. Mean- 
while, neutron scattering measurements suggested that 
the spin correlation length stayed short even at low tem- 
perature To account for such peculiar experimen- 
tal results, Tusnetsugu and Arikawa |j| , Lauchli et al 0| 
and Bhattacharjee et al Q proposed a scenario where the 



spin nematic order, either ferroquadratic (FQ) or antifer- 
roquadratic (AFQ), play a dominant role. Their theoret- 
ical anlyses were performed on the basis of the AF 5 = 1 
Heisenberg model with the blinear-biquadratic exchange. 
Experimentally, a weak but clear anomaly, possibly orig- 
inated from some kind of phase transition, is observed in 
the susceptibility at T ~ 8.5K 0. In the present letter, 
we address the issue of the nature of the experimentally 
observed transition-like anomaly of NiGa2S4. 

The model considered is the 5 = 00 version of the 
5 = 1 Hamiltonian used in Refs.0, @, 0], i.e., a classi- 
cal Heisenberg AF on the 2D triangular lattice with the 
bilinear-biquadratic exchange described by 

H = -JJ2 ^ -Sj-K^iSi -5,) 2 , (1) 

<ij> <ij> 

where J < is the antiferromagnetic bilinear exchange, 
K the biquadratic exchange which is either FQ [K > 0) 
or AFQ (K < 0), and the sum is taken over all nearest- 
neighbor pairs on the lattice. While the biquadratic term 
is essential in stabilizing a hypothetical spin nematic or- 
der, its significance in real systems has not been estab- 
lished yet. The biquadratic term is usually small, while it 
has been argued that it could be large near the Mott tran- 
sition or due to the effect of orbitals p|. In the present 
letter, following Refs.[l,@,[7|, we assume (1), and investi- 
gate its finite-temperature ordering properties by means 
of Monte Carlo (MC) simulations. 

MC simulations are performed based on the standard 
heat-bath method. The system studied is of size Lx L, L 
being in the range from 48 to 192, with periodic boundary 
conditions. The system is gradually cooled from the high 
temperature, each run containing (3 ~ 6)xl0 5 Monte 
Carlo steps per spin (MCS) at each temperature. Aver- 
ages are then made over 5 ~ 10 independent runs. 

Pure bilinear case K = 

In the case of the bilinear interaction only (K — 0), the 
ordering property of the model were studied extensively 
0, d, [if. Numerical studies suggested that the model 
exhibited a Z2 vortex-induced topological transition at 
T = Ty ~ 0.3 (in units of J|), at which the spin corre- 
lation length remains finite. The specific peak exhibits 



2 



a rounded peak above Ty, while no appreciable anomaly 
is observed at Ty. The transition manifests itself as a 
dynamical anomaly 0, E3] • 

A convenient quantity characterizing such a vortex 
transition might be the vorticity modulus, which mea- 
sures the stiffness of the system against spin deformation 
corresponding to vortex formation The vorticity 



modulus is defined by v — AF/ In L where AF is the 
free-energy cost due to the introduction of an isolated 
vortex into the system. In MC simulations, v can be cal- 
culated from appropriately defined fluctuations [||. In 
the vortex-unbounded phase, the system does not ex- 
hibit macroscopic stiffness against vortex formation with 
V = 0, while, in the vortex-bounded phase, the system 
becomes stiff against vortex formation and v > 0. 

Our MC result of the vorticity modulus is shown in 
Fig. la. The data indicate the occurrence of a vortex- 
induced topological transition at T ~ 0.28, consistently 
with the previous results 0, B 0] • 

Ferroquadrapolar case K > 

Next, we analyze the FQ case with K > 0. The 
ground-state of three spins on a triangle is the 120° struc- 
ture for K < 2/9 (measured in units of |J|), whereas 
at K = 2/9 it exhibits a discontinuous change into the 
collinear state with up-up-down (down-down-up) state 
as illustrated in Fig. 2a, which remains to be the ground 
state up to K = oo. Such a collinear ground state re- 
sembles the one of the triangular Ising AF, although in 
the present Heisenberg case the axis of spin collinearity 
can be arbitrary. In the collinear ground state, whether 
each spin points either up or down is not uniquely deter- 
mined due to the frustration-induced local degeneracy: 
See Fig. 2a. Such a local degeneracy leads to a macro- 
scopic degeneracy in an infinite triangular lattice. In- 
deed, one sees from exact information about the cor- 
responding Ising model that the collinear ground state 
does not possess a true AF LRO, but only a quasi-LRO 
with power-law-decaying spin correlations 12|. Mean- 
while, since spins are aligned all parallel or antiparal- 
lel selecting a unique axis in spin space, the collinear 
ground state is characterized by the FQ LRO. The or- 
der parameter of the FQ state is a director, rather than 
the spin itself. In terms of a local quadrapole variable, 
Qinv = SipSiv - (1/3)8 pv, the FQ order parameter Q F 
might be defined by, 
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where < • • • > represents a thermal average. 

In Fig. 3a, we show for the case of K = 0.5 the temper- 
ature dependence of Qf together with that of the Fourier 
magnetization to/ defined by 



(to/) 2 = 2 < \m(q)\ 2 >, 
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FIG. 1: The temperature and size dependence of the vorticity 
modulus for (a) K = 0, (b) K = 0.5, and (c) K = —3. 



where q = (-y,0). Although both Qp and to/ vanish 
in the thermodynamic limit at any T > 0, one can still 
get useful information about the short-range order (SRO) 
from the corresponding finite-size quantities. As can be 
seen from Fig. 3a, the FQ SRO develops rather sharply 
at T ~ 0.4, while the standard AF SRO is kept smaller. 

The Z2 vortex based on the noncollinear spin order is 
expected to survive at least up to K = 2/9. Different 
situation, however, is expected for K > 2/9 since the 
ground state changes from the 120° structure to the FQ 
state. Interestingly, one sees that the FQ state also sus- 
tains a topologically stable Z2 vortex with a parity-like 
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FIG. 2: (a) Frustrated spins on a triangle in the FQ state, 
(b) Z2 vortex formed by directors in the FQ state. 
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Antiferroquadrapolar case K < 

In the case of the AFQ coupling K < 0, the ground 
state of three spins on a triangle remains to be a 120° 
spin structure for K > —1, whereas for K < —1 it takes 
a non-coplanar structure with an angle between two spins 
9 equal to cos 6 = 1/ (2K) . The change in the spin con- 
figuration at K = — 1 is continuous. For K < — 1, due 
to the non-coplanarity of the spin structure, the ground 
state possesses two distinct "chiral" states with mutu- 
ally opposite signs of the scalar chirality S a • Sb X S c . 
This local chiral degeneracy has important consequence 
on the property of an infinite lattice, as the sign of the lo- 
cal chirality tends to take random spatial pattern in the 
ground state, destroying the three-sublattice AF LRO. 
As we shall see below, such a ground state still can sus- 
tain the AFQ order with the three-sublattice periodicity. 

In Fig. 4a, we show a typical snapshot of spin direc- 
tions observed at a temperature T = 0.17, where a typi- 
cal three-sublattice AFQ pattern is realized, with the A-, 
B- and C-sublattice spins pointing to, say, ±S X , ±S y and 
±S Z directions with equal probability. Since such a lo- 
cally orthogonal spin structure is not a ground state for 
| if | < oo, its stabilization should be an entropic effect. 
In Fig. 4b, we show a typical snapshot of spin directions 
at a lower temperature T = 0.01, where a non-orthogonal 
AFQ state is realized in which spins on each triangle lo- 
cally satisfy the above-mentioned ground-sate condition. 
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FIG. 3: The temperature and size dependence of the FQ 
and AFQ order parameters, Qf and Qaf, and the Fourier 
magnetization m/ for the cases of (a) K = 0.5 and (b) K = 
-3. 

topological quantum number (l3l |. A typical example of 
such Zi vortex is illustrated in Fig. 2b: It corresponds to 
a 7r turn (tt disclination) of the director vector. 

Fig. lb exhibits the vorticity modulus for K = 0.5. As 
can be seen from the figure, a vortex-induced topological 
transition occurs at TV — 0.37 in the temperature region 
where the FQ SRO has been developed. Here note the 
difference in the size dependences of v and of Qf (or 
to/) at low temperatures: With increasing L at T 5 Ty, 
while v tends to increase slightly tending to a nonzero 
value, Qf or to/ tends to decrease. Each size dependence 
corresponds to the LRO and the SRO, respectively. 



FIG. 4: Snapshots of spin directions mapped onto a unit 
sphere in spin space for K = —3, at temperatures (a) T = 0.17 
and (b) T = 0.01. Each color represents each sublattice. 

In Fig. 3b, we show to/ and the AFQ order parameter 
Qaf defined by 

(Qaf) 2 =3 £ < (1 ]T q^e^f >, (4) 

[i,i/=x,y,z i 

for the case of A" = —3. The AFQ SRO turns out to de- 
velop rather sharply at T ~ 0.55, where the standard AF 
SRO is still kept small. The AF SRO grows at a lower 
temperature T ~ 0.15. The orthogonal AFQ spin struc- 
ture illustrated in Fig. 4a is realized in the intermediate 
temperature range 0.55 C T 0.15, whereas the non- 
orthogonal AFQ state illustrated in Fig. 4b is realized in 
the lower temperature range T % 0.15. 
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In the AFQ state, the order-parameter space is isomor- 
phic to that of biaxial nematics. The topological defect 
structure of biaxial nematics has been analyzed [l3| : It 
sustains a vortex whose topological quantum number is 
given by the quarternion group, or more precisely, its 
five conjugacy classes. In addition to the vortex-free 
state, there are four topologically distinct vortices. In- 
terestingly, the quarternion group is non-Abelian, which 
might lead to a glassy dynamics via a peculiar combina- 
tion rule of vortices. Even in such an exotic case, the vor- 
tex binding-unbinding mechanism is expected to operate, 
i.e., one expects a vortex-induced topological transition. 

Fig.lc exhibits the vorticity modulus for K = —3. As 
can be seen from the figure, a vortex-induced topological 
transition takes place at Ty ~ 0.5 in the temperature 
region where the AFQ SRO order is developed but the 
magnetic SRO is kept suppressed. In contrast to the K > 
case, the vorticity modulus exhibits a second anomaly 
around a temperature T 2 ~ 0.15 considerably lower than 
the vortex transition temperature. Details of this second 
transition (or crossover) remains to be elucidated. 

Implications to NiGa^Si 

Based on our finding that the bilinear-biquadratic tri- 
angular Heisenberg AF exhibits a vortex-induced topo- 
logical transition, we wish to discuss its possible impli- 
cations to NiGa2S4. We argue that the experimentally 
observed "transition" of NiGa2S4 might be originated 
from a vortex-induced topological transition. The rel- 
evant vortices could be (i) Z 2 vortices based on the non- 
collinear AF order for smaller \K\, (ii) Z 2 vortices based 
on the FQ order for largely positive K, and (iii) quar- 
ternion vortices based on the AFQ order for largely nega- 
tive K. Whichever situation (i)-(iii) applies, the scenario 
immediately explains the experimental observation that 
the spin correlation length remains finite even at and be- 
low the transition. The specific heat is expected to show 
no appreciable anomaly at the transition, only a rounded 
peak above it, which seems consistent with experiments. 
Recent experiments on the nonmagnetic impurity effect 
have revealed that, as the impurity concentration is re- 
duced toward the pure limit, the extent of the spin-glass- 
like hysteretic behavior is suppressed, while the freezing 
temperature Tf itself increases pj. This observation is 
also consistent with our view that the topological transi- 
tion intrinsic to the pure system induces a spin-glass- like 
freezing in the corresponding impure system. 

The next question is obviously which type of vortex is 
relevant in NiGa2S4. Very recent NQR and fiSK mea- 
surements indicate that static internal fields set in below 
Tf accompanied with a divergent increase of the correla- 
tion time toward Tf, at least within experimental time 
window 14| . This observation of internal fields appears 
compatible only with the case (i) above. In the case (i), 
the low-temperature phase should be dominated by spin- 
wave excitations: It is a near critical phase characterized 



by large but still finite spin correlation length and corre- 
lation time. Then, spin waves would be responsible for 
the T 2 specific heat. Indeed, Fujimoto recently accounted 
for the T 2 specific heat based on the spin-wave excita- 
tions of the noncollincar AF order of the S = 1 quan- 
tum magnets, neglecting the vortex degrees of freedom 
flil ]. Vortex- free assumption of Ref. 15 1 is well justified 
at T < Ty, if there occurs a topological transition. Note 
that, in this vortex scenario, the correlation time does 
not truly diverge at Tv(=Tf), but only grows sharply at 
Ty exceeding the experimental time scale, and stays long 
in a wide temperature range below Ty. Such a near criti- 
cal behavior realized below Ty seems consistent with the 
NQR observation [3] • One may suspect that a weak in- 
terplane coupling J', which should exist in real NiGa 2 S4, 
inevitably induces the 3D AF LRO immediately below 
Ty. However, this is not necessarily the case: If J' is suf- 
ficiently small satisfying J'£(Ty) 2 < k B T v , £(Ty) being 
the spin correlation length at Ty, the 3D AF LRO needs 
not set in even below Ty . Finiteness of £ and smallness 
of J 1 are essential in preventing the vortex ordered state 
from forming the 3D AF LRO. At still lower tempera- 
tures, £ diverges exponentially toward T = 0, eventually 
leading to the onset of the magnetic LRO at a certain 
temperature T 1 < Ty. 

In NiGa2S4, distant neighbor interactions neglected in 
the present analysis, particularly the third-neighbor in- 
teraction, compete with the nearest-neighbor one lead- 
ing to an incommensurate spin structure at low temper- 
ature 0. We note that the vortex transition discussed 
here is not specific to the 120° spin structure realized in 
the nearest-neighbor model, but generically expected for 
the noncollincar spin order including the incommensu- 
rate one, although details of the transition needs to be 
clarified further. The vortex scenario might also apply to 
the S=3/2 triangular AF NiCr0 2 0. 

Finally, the noncollincar AF order might explain an- 
other noticeable feature of experiments that the T 2 spe- 
cific heat is quite robust against applied magnetic fields 
1]. It is because the noncollinear AF ground state in 
magnetic fields is capable of keeping an accidental de- 
generacy not related to the Hamiltonian symmetry, es- 
sentially of the same amount as in the zero-field case 
flil ]. Hence, at the classical level, this accidental de- 
generacy gives rise to pseizdo-Goldstone modes even in 
applied fields, which may account for the robustness of 
the low-temperature specific heat, while this degeneracy 
would become approximate in quantum systems. 

In summary, we studied the ordering properties of the 
AF Heisenberg model on the triangular lattice with the 
the bilinear-biquadratic coupling, and have shown that 
the model exhibits a vortex-induced topological transi- 
tion. The relevant vortices could be of three different 
types, depending on the value of the biquadratic cou- 
pling. It was then suggested that the peculiar phase 
transition recently observed in NiGa 2 S4 might have its 
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origin in such a vortex-induced topological transition. 
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